Flexible data organization for images

ABSTRACT

A system, a method and computer-readable media for encoding image data into a compressed bitstream. A mode selection component is configured to select a mode of operation for use in encoding image data. A spatial mode encoder is utilized when the mode selection component selects a spatial mode of operation. The spatial mode encoder is configured to encode the image data into packets by organizing the image data in accordance with a spatial partitioning. A frequency mode encoder is utilized when the mode selection component selects a frequency mode of operation. The frequency mode encoder is configured to encode the image data into packets by organizing the image data in accordance with a frequency ordering.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of a prior application titled“Flexible Data Organization for Images” (Ser. No. 11/444,685, filed Jun.1, 2006 now U.S. Pat. No. 7,949,054), priority from the filing date ofwhich is hereby claimed under 35 U.S.C. §120.

BACKGROUND

In recent years, computer users have become more and more reliant uponpersonal computers to store and present a wide range of digital media.For example, users often utilize their computers to store and interactwith digital images. A digital image is a data entity composed of one ormore two dimensional planes of data. An uncompressed or “source” imageis generally stored or serialized in a raster scan order. To performoperations with respect to the uncompressed image, the pixels arescanned from left to right, top to bottom, and the color channels aregenerally interleaved. Uncompressed digital images, however, can consumeconsiderable storage and transmission capacity.

To more efficiently store and transmit digital images, the image datamay be compressed, and a variety of compression techniques exist in theart. In a compressed domain, data organization is determined, in part,by the compression algorithm. Image compression techniques generallyremap the image into an alternate space by means of a data transform.For instance, data transforms such as discrete cosine transforms (DCT)and wavelets may be used to compress data. Compressed data may beorganized in blocks having sequences of transform coefficients. Theblocks may be scanned in order (e.g., left to right, top to bottom),with in-block data scanned using a “zig-zag scan pattern.” In analternative organization, the information related to the DC coefficientsis serialized first, and the remaining coefficients follow.

Existing compression techniques may offer multiple degrees of freedom inorganizing the transform data. For instance, data can be serialized byfrequency band, by “precinct” (i.e., a spatial partitioning of data), bybitplane and/or by color channel. While this organization providesfreedom to a user, it may be difficult to implement. For example,decoders attempting to recreate images may require additionalcapabilities to handle the various combinations of data. Further,decoders may require more memory and more computational power to handlearbitrarily organized data. In short, while existing compressiontechniques may provide flexibility in the organization of compresseddomain data, an unacceptably high level of complexity often accompaniessuch flexibility. Indeed, existing compression techniques do not offer aflexible, yet tractable, organization of compressed domain data.

SUMMARY

The present invention meets the above needs and overcomes one or moredeficiencies in the prior art by providing systems and methods fororganizing a bitstream of compressed data. A mode selection component isconfigured to select a mode of operation for use in encoding image data.A spatial mode encoder is utilized when the mode selection componentselects a spatial mode of operation. The spatial mode encoder isconfigured to encode the image data into packets by organizing the imagedata in accordance with a spatial partitioning. A frequency mode encoderis utilized when the mode selection component selects a frequency modeof operation. The frequency mode encoder is configured to encode theimage data into packets by organizing the image data in accordance witha frequency ordering.

It should be noted that this Summary is provided to generally introducethe reader to one or more select concepts described below in theDetailed Description in a simplified form. This Summary is not intendedto identify key and/or required features of the claimed subject matter,nor is it intended to be used as an aid in determining the scope of theclaimed subject matter.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

The present invention is described in detail below with reference to theattached drawing figures, wherein:

FIG. 1 is a block diagram of a network environment suitable for use inimplementing the present invention;

FIG. 2 is a block diagram illustrating an image that is partitioned inaccordance with one embodiment of the present invention;

FIG. 3 illustrates a method in accordance with one embodiment of thepresent invention for reconstructing an image from a compressedbitstream;

FIG. 4 is a block diagram illustrating the decoding of tiles inaccordance with one embodiment of the present invention;

FIG. 5 is a schematic diagram illustrating a system for reconstructingan image from a compressed bitstream in accordance with one embodimentof the present invention;

FIG. 6 illustrates a method in accordance with one embodiment of thepresent invention for transcoding a cropped version of a digital image;

FIG. 7 is a block diagram illustrating an image undergoing croppingoperations in accordance with one embodiment of the present invention;

FIGS. 8A, 8B and 8C are block diagrams of a bitstream in accordance withone embodiment of the present invention;

FIG. 9 is a schematic diagram illustrating a system for encoding abitstream in accordance with one embodiment of the present invention;

FIG. 10 illustrates a method in accordance with one embodiment of thepresent invention for decoding image data from a bitstream; and

FIG. 11 is a block diagram illustrating the packetization and indexingof a bitstream in accordance with one embodiment of the presentinvention.

DETAILED DESCRIPTION

The subject matter of the present invention is described withspecificity to meet statutory requirements. However, the descriptionitself is not intended to limit the scope of this patent. Rather, theinventors have contemplated that the claimed subject matter might alsobe embodied in other ways, to include different steps or combinations ofsteps similar to the ones described in this document, in conjunctionwith other present or future technologies. Moreover, although the term“step” may be used herein to connote different elements of methodsemployed, the term should not be interpreted as implying any particularorder among or between various steps herein disclosed unless and exceptwhen the order of individual steps is explicitly described. Further, thepresent invention is described in detail below with reference to theattached drawing figures, which are incorporated in their entirety byreference herein.

Referring initially to FIG. 1 in particular, an exemplary operatingenvironment for implementing the present invention is shown anddesignated generally as computing device 100. Computing device 100 isbut one example of a suitable computing environment and is not intendedto suggest any limitation as to the scope of use or functionality of theinvention. Neither should the computing-environment 100 be interpretedas having any dependency or requirement relating to any one orcombination of components illustrated.

The invention may be described in the general context of computer codeor machine-useable instructions, including computer-executableinstructions such as program modules, being executed by a computer orother machine, such as a personal data assistant or other handhelddevice. Generally, program modules including routines, programs,objects, components, data structures, etc., refer to code that performparticular tasks or implement particular abstract data types. Theinvention may be practiced in a variety of system configurations,including hand-held devices, consumer electronics, general-purposecomputers, specialty computing devices (e.g., cameras and printers),etc. The invention may also be practiced in distributed computingenvironments where tasks are performed by remote-processing devices thatare linked through a communications network.

With reference to FIG. 1, computing device 100 includes a bus 110 thatdirectly or indirectly couples the following elements: memory 112, acentral processing unit (CPU) 114, one or more presentation components116, input/output ports 118, input/output components 120, anillustrative power supply 122 and a graphics processing unit (GPU) 124.Bus 110 represents what may be one or more busses (such as an addressbus, data bus, or combination thereof). Although the various blocks ofFIG. 1 are shown with lines for the sake of clarity, in reality,delineating various components is not so clear, and metaphorically, thelines would more accurately be gray and fuzzy. For example, one mayconsider a presentation component such as a display device to be an I/Ocomponent. Also, CPUs and GPUs have memory. The diagram of FIG. 1 ismerely illustrative of an exemplary computing device that can be used inconnection with one or more embodiments of the present invention.Distinction is not made between such categories as “workstation,”“server,” “laptop,” “hand-held device,” etc., as all are contemplatedwithin the scope of FIG. 1 and reference to “computing device.”

Computing device 100 typically includes a variety of computer-readablemedia. By way of example, and not limitation, computer-readable mediamay comprise Random Access Memory (RAM); Read Only Memory (ROM);Electronically Erasable Programmable Read Only Memory (EEPROM); flashmemory or other memory technologies; CDROM, digital versatile disks(DVD) or other optical or holographic media; magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,carrier wave or any other medium that can be used to encode desiredinformation and be accessed by computing device 100.

Memory 112 includes computer-storage media in the form of volatileand/or nonvolatile memory. The memory may be removable, nonremovable, ora combination thereof. Exemplary hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 100includes one or more processors that read data from various entitiessuch as memory 112 or I/O components 120. Presentation component(s) 116present data indications to a user or other device. Exemplarypresentation components include a display device, speaker, printingcomponent, vibrating component, etc.

I/O ports 118 allow computing device 100 to be logically coupled toother devices including I/O components 120, some of which may be builtin. Illustrative components include a microphone, joystick, game pad,satellite dish, scanner, printer, wireless device, etc.

As previously mentioned, the present invention provides an improvedsystem and method for representing digital images. An image, as the termis used herein, is composed of multiple color planes (or a singleplane). The first color plane is referred to as luminance and roughlycorresponds to a monochrome representation of the image. The remainingcolor planes are referred to as chrominance planes. Generally, luminanceand chrominance planes are of the same size. However, in certaininstances, the chrominance planes may be half the width and/or height ofthe luminance planes. In addition to the luminance and chrominanceplanes, an image may carry an alpha plane that carries transparencyinformation.

FIG. 2 illustrates an image 200 that is partitioned in accordance withan image structure hierarchy. The image 200 is composed ofnon-overlapping four pixel by four pixel blocks. Though these blocks arenot presented on the FIG. 2, the entire area of the image 200 may bedivided into such blocks. The blocks cover the entire image and mayspill over the image boundaries. In this case, pixel values within ablock but outside of the image are arbitrarily defined by extension orextrapolation. In one embodiment, the blocks are equally applicable toluminance, chrominance and alpha planes regardless of color format. Inthis embodiment, blocks of all colorplanes are collocated for colorformats other than grayscale, YUV 4:2:0 and YUV 4:2:2. For blocks of YUV4:2:0 and YUV 4:2:2, only the chrominance planes are collocated.

The blocks may be grouped into non-overlapping four block by four blockclusters, known as macroblocks. For example, a macroblock 202 resideswithin the image 200. Though only one macroblock is presented on theFIG. 2, the entire area of the image 200 may be divided into thesemacroblocks. Each macroblock contains blocks of all colorplanes, and themacroblocks form a regular pattern on the image 200. For example,macroblocks of grayscale images are composed of sixteen blocks, whilemacroblocks of YUV 4:2:0 images are composed of sixteen luminanceblocks, and four each of U and V plane blocks in a two-by-two patterncollocated with the sixteen luminance blocks. Macroblocks of YUV 4:2:2images are composed of sixteen luminance blocks, and eight each of U andV plane blocks in a four-by-two pattern collocated with the sixteenluminance blocks. Macroblocks of YUV 4:4:4 images are composed offorty-eight blocks, sixteen for each plane in collocated four-by-fourpatterns. Like blocks, the macroblocks may spill over image boundaries.In this case, pixel values outside of the image and within a macroblockare arbitrarily defined such as by extension or extrapolation. Forexample, the dashed-lines 204 and 206 represent the original right andbottom edges of the image 200. To align the edges of the image 200 withthe edges of the various macroblocks, the image 200 includesextrapolated values for pixels residing beyond the dashed-lines 204 and206.

The macroblocks may be grouped into regular structures called tiles. Forexample, tile boundaries 208, 210, 212 and 214 divide the image 200 intoeight tiles. The tiles extend to the edges of the image 200 and form aregular pattern on the image 200. Stated another way, the tiles in ahorizontal row are of the same height and are aligned, while the tilesin a vertical column are of the same width and are aligned. Subject tosuch a regular pattern, the tiles may be of arbitrary size, but theymust be macroblock aligned. So the tiles extend beyond the dashed-lines204 and 206 to ensure macroblock alignment. In one embodiment, the image200 contains between one and 256 columns of tiles in the horizontaldirection and between one and 256 rows of tiles in the verticaldirection. In this embodiment, the image 200 may contain between one and65536 tiles. When an image contains one tile, it is said to be“untiled.” If the number of tiles is greater than one, the image is saidto be “tiled.”

As will be appreciated by those skilled in the art, spatial partitioningof an image may be desirable for several reasons. For one, spatialpartitioning divides the image into independently decodable regions, andthese regions can be sent in the bitstream in an arbitrary order, or notat all. This allows for the efficient extraction of regions and for theability to encode and decode large images with the traditional sizebounds.

FIG. 3 illustrates a method 300 for reconstructing an image from acompressed bitstream. The method 300, at 302, accesses the compressedbitstream. The compressed bitstream may define a spatial partitioningfor the image. For example, the image may be partitioned in accordancewith the image structure hierarchy illustrated by FIG. 2. The method 300may use a partition rule in the signaling of the partition information.This partition rule may be defined in the bitstream subject to formatspecifications.

In one embodiment, the method 300 employs a simple partitioning rule—theimage is divided into non-overlapping rectangular regions (known astiles) such that all partitions along a row of pixels have the sameheight and all partitions along a column of pixels have the same width.Further, all partitions are aligned with macroblocks, meaning that thegrid is aligned to multiples of 16 pixels in both vertical andhorizontal directions. Such a partitioning yields the previouslydiscussed tiles of FIG. 2, and this partition rule may be communicatedin the bitstream in a very concise manner. First, the size of the imageis sent in the bitstream. From this, the width and height in macroblockunits can be derived as (x+15)/16, where x is the image size (width orheight) and integer division is used. Second, the number of horizontaland vertical tiles is sent in the bitstream. Next, the width (inmacroblock units) of each of the horizontal and vertical partitioning(except the last one) is sent. This information may be sent with a fixedor variable length code.

At 304, the method 300 decodes a portion of the bitstream that isassociated with a tile. For example, the information in the bitstreamrelated to a tile may be decoded to yield a sequence of transformcoefficients and/or other meaningful symbols. As previously mentioned,tiling is a spatial partitioning of the image into “independentlydecodable regions.” In the bitstream, all the compressed data related toa spatial tile is located in an addressable and contiguous area or areas(often referred to as packets) in a file. Therefore, by decoding thesepackets the tile may be reconstructed, subject to compression loss.

To minimize or eliminate such compression loss, the method 300, at 306,obtains additional transform coefficients by decoding data associatedwith areas of the image that are adjacent to the tile of interest. Asknown to those in the art, strict data partitioning with independentlydecodable tiles may give rise to sharp edge artifacts between tiles. Analternate goal of tiling, however, is for a tile to be independentlydecodable in an entropy coding sense but not in a signal processingsense. Entropy coding, in general, is known in the art. Using entropycoding, reconstructed data within a tile depends on the transformcoefficients of neighboring blocks and macroblocks adjacent toboundaries of the tile of interest. Therefore, the method 300 obtainssuch additional transform coefficients at 306.

At 308, the method 300 reconstructs the portion of the image associatedwith the tile of interest. The reconstructed data depends on transformcoefficients and other information not only from the tile of interest,but the data also depends on transform coefficients from the other,adjacent tiles. As will be appreciated by those skilled in the art,images reconstructed in this manner will not visibly indicate evidenceof tiling, assuming that a suitable decorrelating transform is appliedacross block boundaries and reasonable encoder parameters are used.

FIG. 4 provides a schematic diagram of a system 400 for independentlydecoding tiles in an entropy sense (i.e., by the sharing of entropydecoded information across tiles). Data associated with a first tile isdecoded into transform coefficients at a block 402. These transformcoefficients are used by a transform block 404 to reconstruct theportion of an image associated with the first tile.

To reconstruct the first tile, the transform block 404 also utilizestransform coefficients associated with a second tile. The second tile isadjacent to the first tile. Data associated with the second tile isdecoded into transform coefficients by a block 406. Transformcoefficients from the block 406 are shared with the transform block 404to aid in reconstructing the first tile. Likewise, a transform block 408reconstructs the second tile with transform coefficients from both theblock 402 and the block 406 (i.e., coefficients from both the first andthe second tiles).

To reconstruct the second tile, the transform block 408 also utilizestransform coefficients associated with a third tile. The third tile isadjacent to the second tile. Data associated with the third tile isdecoded into transform coefficients by a block 410. These transformcoefficients from the block 410 are then shared with the transform block408 to aid in reconstructing the second tile. Similarly, a transformblock 412 reconstructs the third tile with transform coefficients fromboth the block 406 and the block 410. FIG. 4 depicts a ID embedding oftiles. In two dimensions, a tile may need to access data in up to 8other tiles adjacent to it in order to reconstruct the tile.

It should be noted that the system 400 represents a decoder. As will beappreciated by those skilled in the art, encoder side tiling is a mirrorimage of decoder side tiling. Those skilled in the art will furtherappreciate that tiles may be entropy decoded in arbitrary order, and,subsequent to gathering all the relevant data, transforms can be appliedin any order as well. In one embodiment, the shared data illustrated byFIG. 4 is only one macroblock wide. Since the shared information isrelatively small, tiles may be treated as strict tiles on the decodeside at a small cost of decreased visual quality. This may be necessarywhen sharing of data is not feasible.

FIG. 5 illustrates a system 500 for reconstructing an image from acompressed bitstream. The system 500 includes a bitstream accesscomponent 502. The bitstream access component 502 is configured toaccess a compressed bitstream that includes encoded image data. In oneembodiment, the compressed bitstream defines a spatial partitioning foran image that divides the image into one or more tiles, i.e., one ormore independently decodable regions.

The system 500 further includes a decoding component 504, which isconfigured to generate a series of transform coefficients by decodingthe encoded image data from the compressed bitstream. For example, thedecoding component 504 may decode the image data associated with aselected tile. Further, the decoding component 504 may decode image dataassociated with tiles adjacent to the selected tile.

An image reconstruction component 506 is also included in the system500. The image reconstruction component 506 may be configured toreconstruct the image from the transform coefficients. To reconstruct atile, the image reconstruction component 506 may utilize both transformcoefficients associated with the selected region and transformcoefficients associated with one or more areas adjacent to the selectedregion. For example, the image reconstruction component 506 may utilizethe system 400 of FIG. 4 for independently decoding tiles in an entropysense

The system 500 may optionally include an encoding component 508 and/or auser interface component 510. The encoding component 508 may beconfigured to utilize entropy encoding to encode image data. Suchencoding is the mirror image of the decoding performed by the decodingcomponent 504. This encoding utilizes image data associated with areasadjacent to a selected region in the encoding of that region. Finally,the user interface component 510 may be configured to enable the imageto be presented to a user after the image has been reconstructed.

As will be appreciated by those skilled in the art, one desirablefeature of a good compressed image format is the ability to losslesslytranscode a cropped version of the image within the same format.Lossless in this case means there is no additional loss introduced inthe process of cropping. The format described herein may supports thisfeature, and FIG. 6 illustrates a method 600 for transcoding a croppedversion of a digital image.

The method 600, at 602, receives a selection of a crop area within adigital image. The crop area may be selected by a variety of inputs orsignals. While the original digital image may be partitioned intomacroblocks, the crop area may not align with the macroblock grid. Inthis instance, macroblocks in the original image will not directlycorrespond to macroblocks in the cropped image. Even if the crop area isfully macroblock aligned, the cropped area will depend on macroblockdata outside of the cropped area due to overlap. This means that theneighboring macroblock data must be retained along with the croppedimage.

At 604, the method 600 extends the crop area by a predetermined amountin at least one direction. The new area may be referred to as anextended crop area. In one embodiment, the extended crop area isobtained by stretching the crop area by a predetermined number of pixelsin each direction, except where the side runs into the boundary of theimage.

The method 600, at 606, identifies each macroblock that overlaps atleast a portion of the extended crop area. The macroblocks are obtainedby macroblock aligning the extended crop area in all directions so as toinclude all macroblocks overlapping the extended crop area. Theseidentified macroblocks may be referred to as the extended crop image. At608, the method 600 transcodes the identified blocks to generate acropped version of the digital image. In one embodiment, this area istranscoded by entropy decoding the relevant tiles and entropy encodingthe relevant macroblocks alone. Indeed, the tiling structure of theextended cropped image is derived from the tiling structure of theoriginal image by default. Since bitstream parameters can change at thetile level, the tiles cannot be coalesced in general.

FIG. 7 illustrates an image 700 undergoing cropping operations. A croprectangle 702 is identified within the image 700. In one embodiment, thecrop rectangle 702 is permitted to be non-macroblock aligned in alldirections. The crop rectangle 702, for example, may be inscribed bythree pixels on the top, left, right and bottom of the macroblock grid.This allows the crop rectangle 702 to be arbitrarily aligned. Tofacilitate this arbitrary alignment, an extended signaling mechanism maybe used in the image header of the new “cropped” image defined by thecrop rectangle. For example, a flag may indicate whether the croppedimage is inscribed. If the crop rectangle 702 is inscribed, symbols maybe sent to indicate the inscribed offsets in the top, left, bottom andright directions.

The image 700 further includes an extended crop rectangle 704. Theextended crop rectangle 704 may be obtained by stretching the croprectangle 702 by D pixels in each direction, except where a side runsinto a boundary of the image 700. In one embodiment, D is set to 0, 2 or10 pixels for three permissible overlap modes that correspond to nooverlap, overlap of high frequency alone and overlap of both low andhigh frequencies.

An extended cropped image 706 is also included in the image 700. Theextended cropped image 706 is obtained by macroblock aligning theextended crop rectangle 704 in all directions. Such macroblock aligningallows selection of all macroblocks overlapping the extended croprectangle 704. The extended cropped image 706 may be transcoded byentropy encoding the relevant macroblocks alone. Inscribed offsets mayalso correspond to the extended cropped image 706.

FIGS. 8A, 8B and 8C illustrate a bitstream 800 that contains informationrepresenting a digital image. Turning initially to FIG. 8A, thebitstream 800 includes an image header 802 and an index table 804. Theseelements may contain a variety of information enabling the decoding ofthe bitstream 800. Following the index table 804, the bitstream 800includes a packet 806 and a packet 808. In one embodiment, each tile ofan image is placed in a separate data segment called a packet, and thebitstream 800 is organized in accordance with such packets. Accordingly,the packets 806 and 808 each correspond to an image tile.

In one embodiment, there are two fundamental modes of operationaffecting the structure of the bitstream—a spatial mode and a frequencymode. A one-bit signal in the image header 802 specifies which of themodes is used. In both the modes, the bitstream is laid out as a header,followed by a sequence of tiles as shown in FIG. 8A.

FIG. 8B illustrates the structure of the packet 808 when the spatialmode is in operation. In the spatial mode, the bitstream of each tile islaid out in macroblock order, and the compressed bits pertinent to eachmacroblock are located together. Accordingly, the packet 808 includessegments 810, 812, 814 and 816. Each of these segments is associatedwith one macroblock. In one embodiment, the macroblock data is laid outin raster scan order, i.e. scanning left to right, top to bottom.

FIG. 8C illustrates the structure of the packet 808 when the frequencymode is in operation. In the frequency mode, the bitstream of each tileis laid out as a hierarchy of bands. The first band 818 carries the DCvalue of each macroblock in raster scan order. The second band 820carries the lowpass coefficients. For example, there may be fifteenlowpass coefficients in each macroblock for each color plane, with somepossible exceptions. The third band 822 carries the remaining high passcoefficients of each macroblock colorplane (with some possibleexceptions). Finally, the fourth band 824 is called Flexbits and is anoptional layer that carries information regarding the low order bits ofthe highpass coefficients particularly for lossless and low loss cases.

In one embodiment, the entire or several lower bits of the Flexbitsbands may be skipped in the bitstream. If the entire Flexbits bands areskipped, the highpass subbands may also be skipped. Furthermore, if thehighpass subbands are skipped, the lowpass subbands may be skipped. Thisapplies to both bitstream modes. (Note that in the spatial mode, eachmacroblock consists of a DC part, a lowpass part, a highpass part and aFlexbits part). In this embodiment, the decoder may have the flexibilityto fill the skipped data, and the skipped segments may be signaled inthe image header with a code. This embodiment may be useful forapplications such as image downsampling and low bitrate coding.

As will be appreciated by those skilled in the art, each mode ofoperation may have its own advantages. The spatial mode, for example,may be easier to implement and less complex, while having a smallermemory footprint on the encoder/decoder. The frequency mode may offerbenefits of a hierarchical representation of the image, which allows forthumbnails, progressive display, fast zooming, and error protection.

FIG. 9 illustrates a system 900 for encoding a bitstream such as thebitstream 800. The system 900 includes a mode determination component902. The mode determination component 902 permits selection of eitherthe spatial mode or the frequency mode. As previously mentioned, thesemodes each have their own advantages. Once a mode is determined, animage may be reduced to a bitstream in accordance with the selectedmode. Accordingly, the system 900 includes a spatial mode encoder 904for use in structuring packet data in accordance with the spatial mode.When the frequency mode is selected, a frequency mode encoder 906 may beused to structure packet data in accordance with the frequency mode.

FIG. 10 illustrates a method 1000 for decoding image data from abitstream. At 1002, the method 1000 accesses a compressed bitstream. Forexample, the compressed bitstream may be the bitstream 800 of FIG. 8.The method 1000, at 1004, determines whether the packets in thebitstream are encoded in accordance with either the spatial mode or thefrequency mode. A variety of techniques may be utilized in thisdetermination. In one embodiment, a signal in an image header specifieswhich mode is present. At 1006, the method 1000 decodes the compressedbitstream in accordance with the mode that was determined at 1004.

FIG. 11 illustrates the packetization and indexing of a bitstream 1100.The bitstream 1100 includes an image header 1102, which contains avariety of information needed to decode the bitstream 1100. For example,the image header 1102 contains a table of pointers referred to as anindex table 1104.

The index table 1104 is optional if the number of tiles is one and thespatial mode is used. Otherwise, the index table 1104 is mandatory. Inone embodiment, the index table 1104 is byte aligned, and the pattern offirst two bytes is hexadecimal 0x000. Following these bytes resides asequence of pointers known as tile pointers 1106. The number of the tilepointers 1106 is equal to the number of tiles if the spatial mode isused and is equal to the number of tiles times (four minus the number ofbands skipped) if the frequency mode is used. Each pointer may be avariable length symbol, and all pointers are non-negative. The indextable 1104 further includes another pointer called the origin pointer1108. The origin pointer 1108 defines an un-accessed segment of data1110 and defines an origin 1112. The pointers in the tile pointers 1106index from the origin 1112 and provide offsets from the origin 1112 torespective tiles. The un-accessed segment of data 1110 is a data segmentnot currently handled but which may be used in bitstream extensions.

The bitstream 1100 further includes image data 1114, which is organizedin accordance with the previously discussed tiling. Each tile in animage is placed in a separate data segment called packet. For example,the image data 1114 includes a first packet 1116, which contains datafor a first tile. Also, the image data 1114 includes a second packet1118, which contains data for a second tile. In one embodiment, thepackets 1116 and 1118 each include a packet header followed by packetdata. The packet header carries information required to decode thepacket data such as quantization modes and parameters. The packet headermay include information related to the tile location and the tile type.For example, the packet header may indicate the tile includes the DCband, the lowpass band, the highpass band or the Flexbits component.

In one embodiment, there may be un-accessed data between the packets1116 and 1118 in the image data 1114. The un-accessed segment of data1110 also represents unused data. These two “gaps” in the bitstreamformat may be intentional. These gaps allow for additional data to beinserted, while maintaining compatibility of the bitstream with acurrent format. As will be appreciated by those skilled in the art,these gaps support backward compatibility by enabling a future,presumably higher quality bitstream to be decoded (to some degree offidelity or viewability) by existing decoders.

Further, the image header 1102 may contain version information for thebitstream 1100. Future bitstreams that are backward compatible with thecurrent format may use a higher version number than the bitstream 1100.As will be appreciated by those skilled in the art, this signalingdesign enables backward compatibility and may allow future decoders toexercise future bitstream changes.

Alternative embodiments and implementations of the present inventionwill become apparent to those skilled in the art to which it pertainsupon review of the specification, including the drawing figures.Accordingly, the scope of the present invention is defined by theappended claims rather than the foregoing description.

1. A system that encodes color-plane data of an image into a packet of acompressed bitstream, said system comprising: a mode-selection componentthat selects a mode of operation from among a plurality of modes,wherein the mode of operation dictates an organization of thecolor-plane data when the color-plane data is encoded into the packetand wherein the color-plane data prescribes a rendering of one or morecolor planes of the image; a spatial-mode encoder that is utilized whenthe mode-selection component selects a spatial mode of operation,wherein said spatial-mode encoder encodes the color-plane data into saidpacket by organizing the color-plane data into ordered macroblockswithin at least a portion of said packet in accordance with a spatialpartitioning; and a frequency-mode encoder that is utilized when themode-selection component selects a frequency mode of operation, whereinsaid frequency-mode encoder encodes the color-plane data into saidpacket by organizing the color-plane data within at least a portion ofsaid packet in accordance with a frequency ordering.
 2. The system ofclaim 1, wherein the ordered macroblocks, when assembled together,comprise an independently decodable region of the image, and wherein thespatial partitioning divides at least a portion of said independentlydecodable region into one or more blocks.
 3. The system of claim 1,wherein said frequency ordering organizes the color-plane dataassociated with the packet into a plurality of bands.
 4. The system ofclaim 3, wherein at least one of said plurality of bands includes DCvalues associated with the packet.
 5. The system of claim 3, wherein atleast one of said plurality of bands includes highpass or lowpasscoefficients associated with the packet.